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INTRODUCTION 


Advances  in  computer  and  communications  technology  have 
enabled  the  development  of  increasingly  complex  systems.  As  a 
result,  the  humans  involved  in  these  systems  are  faced  with 
substantial  increases  in  the  amount  of  information  they  must 
integrate  into  the  decision  making  process  and,  in  some  cases, 
increasingly  tight  time  constraints  within  which  decisions  must 
be  made.  With  goals  of  ameliorating  these  problems,  a  variety  of 
decision  aids  have  been  developed  and  tested.  Unfortunately,  few 
of  the  plethora  of  aids  available  have  met  any  significant  degree 
of  operational  success. 

There  are  several  possible  explanations  for  this  lack  of 
success.  One  possibility  is  that  the  aiding  methodologies 
employed  have  been  inadequate.  Another  possibility  is  that  the 
interface  between  user  and  aid  limits  success  even  if  the 
underlying  methodology  is  appropriate.  Yet  another  potential 
explanation  for  aids  not  being  accepted  is  that  user 
participation  in  design  has  not  been  given  the  necessary  primacy. 
Many  authors  have  argued  for  improvements  in  these  areas  and  a 
general  awareness,  if  not  solutions,  of  these  problems  is 
ubiquitous. 

Another  potential  contributing  factor  is  the  inherent 
inflexibility  of  an  aid  that  only  provides  static  computational 


1 


and  bookkeeping  functions  designed  to  support  the  average  user  in 
the  average  situation,  when  virtually  all  decision  making 
involves  a  particular  individual  in  a  specific  situation.  What 
may  be  required  for  success  is  that  aids  adapt  to  individuals  and 
situations,  perhaps  in  a  manner  similar  to  that  of  human  staff  or 
crew  members  in  the  military  and  industry.  It  seems  reasonable 
to  argue  that  aids  which  could  adapt  in  this  manner  are  quite 
likely  to  overcome  many  of  the  user  acceptance  problems 
encountered  in  the  past. 

The  purpose  of  this  report  is  to  present  a  framework  for 
research  on  adaptive  decision  aids  and  to  illustrate  the  use  of 
this  framework  for  recommending  important  and  promising  research 
directions  within  the  area  of  command  and  control  systems.  The 
discussion  begins  with  an  overview  of  recent  literature  on 
decision  aids,  proceeds  to  presentation  of  the  general  framework, 
and  concludes  with  consideration  of  implications  for  command  and 
control. 


THE  LITERATURE 


In  order  to  assess  the  state  of  the  art  in  decision  aiding, 
an  extensive  literature  review  was  performed.  Emphasis  was 
placed  on  the  journal  and  conference  publications  of  the  IEEE 
Systems,  Man,  and  Cybernetics  Society  as  well  as  government 
contractor  technical  reports.  These  two  sources  produced  104 
documents;  16  additional  documents  were  obtained  from  other 
journals  and  conference  proceedings.  Thus,  the  discussion  in 
this  section  is  based  on  a  survey  of  120  documents.  Considering 
the  fact  that  many  of  these  documents  (i.e.,  20)  were  in 
themselves  reviews  of  the  literature,  the  breadth  of  literature 
covered  both  directly  and  indirectly  is  substantial. 

The  literature  which  focuses  on  particular  applications  can 
be  divided  into  three  categories:  1)  military,  2)  nonmilitary, 
and  3)  methodological.  In  the  military  category,  42  documents 
were  reviewed  in  the  following  subcategories:  Army  tactical 
operations  (12),  Air  Force/Navy  aircraft  operations  (17),  Navy 
marine  operations  (10),  and  other  (3).  The  nonmilitary  category 
included  36  documents  in  the  following  subcategories:  aircraft 
operations  (9),  process  control  and  manufacturing  (8),  management 
(7),  design  and  analysis  (6),  and  other  (6).  The  methodological 
category  included  14  documents  in  the  subcategories  of  decision 
structuring  (9)  and  extensions  of  decision  analysis  (5).  Thus, 
the  distribution  of  documents  across  applications  areas  is 


reasonably  broad. 


Many  of  the  documents  discuss  topics  that  are  fairly  general 
and  go  beyond  any  particular  application.  Six  general  topics 
that  emerge  are: 

1.  Reviews  of  human  information  processing,  behavioral  decision 
theory,  aiding  methodologies,  available  decision  aids,  and 
evaluation  (20). 


2.  Attributes  of  situations,  information,  and  decision  making 
(13). 


3.  Disaggregations  of  decision  making  into  various  tasks, 
phases,  stages,  etc.  (8). 

4.  Types  and  levels  of  decision  aiding  (14). 


5.  Aids  that  adapt,  or  can  be  adapted,  to  particular  situations 
and/or  individual  decision  makers  (29). 


6.  Principles,  guidelines,  and/or  processes  for  designing  aids 

(16). 


(The  number  in  parentheses  following  each  topic  indicates  the 
number  of  documents  discussing  that  topic.) 


Reviews 


Considering  the  review  literature,  the  topic  most  frequently 
addressed  is  human  information  processing.  These  reviews  amass 
an  impressive  array  of  literature  to  support  the  overall 


conclusion  that  human  abilities  are  limited  and  individual 
differences  are  important.  It  is  usually  argued,  quite 
reasonably,  that  human  limitations  should  be  considered  when 
designing  aids  and,  if  possible,  aids  should  adapt  to  individual 
differences. 

Another  popular  topic  in  the  review  literature  is  the 
potential  applicability  to  decision  aiding  of  artificial 
intelligence  or  advanced  software  engineering  methods.  There 
appears  to  be  a  fairly  clear  consensus  that  traditional  decision 
analytic  methods  (i.e.,  decision  trees,  multiattribute  utility 
theory,  etc.)  must  be  augmented  to  achieve  wider  applicability 
and  acceptance.  This  is  particularly  true  in  terms  of  initial 
problem  structuring  and  generation  of  alternatives.  In  general, 
the  rule-based  or  knowledge-based  "expert"  systems  deal  with  the 
context  of  a  problem  more  directly  than  possible  with  decision 
analytic  methods. 

Decision  Making  Tasks 

A  variety  of  authors  have  concerned  themselves  with 
classifying  decision  making  situations,  information  requirements 
and  systems,  and  decision  making  problems.  The  attributes  of  the 
resulting  classification  schemes  tend  to  be  either  rather 
qualitative  or  fairly  context-specific.  All  of  these 

efforts  represent  attempts  to  partition  the 


classification 


decision  aiding  problem  into  more  manageable  subproblems.  As 
will  be  seen,  this  approach  has  been  adopted  in  this  report  for 
formulating  the  overall  problem  of  adaptive  decision  aiding. 


One  aspect  of  this  decomposition  approach  that  deserves 
particular  attention  is  the  disaggregation  of  decision  making 
into  tasks,  phases,  stages,  etc.  While  a  wide  variety  of 
alternative  decompositions  have  been  discussed,  the  differences 
among  these  alternatives  are  mainly  due  to  variations  in 
terminology  among  contexts.  It  is  fairly  straightforward  to 
synthesize  the  alternatives  into  a  single  set  of  decision  making 
tasks  such  as  the  following: 

1 .  Detection  of  problem 

2.  Acquisition  of  information 

3.  Evaluation  of  information 

4.  Integration  of  information 

5.  Generation  of  alternatives 

6.  Evaluation  of  alternatives 

7.  Selection  among  alternatives 

8.  Implementation  of  choice 

9.  Monitoring  of  consequences 


The  tasks  in  this  list  could  be  aggregated  into  decision  making 
functions  such  as  situation  assessment  (nos.  1,  2,  3,  and  4)> 
planning  (nos.  5,  6,  and  7),  and  control  (nos.  8  and  9)  or 
disaggregated  into  elements  of  human  information  processing  such 
as  perception,  estimation,  prediction,  etc.  The  level  of 
description  which  is  most  appropriate  depends  on  the  purpose  of 
the  description.  A  later  section  of  this  report  pursues  this 
issue  further  in  the  context  of  command  and  control  systems. 

Decision  Aiding  Possibilities 


Considering  the  many  documents  which  discuss 
levels  of  decision  aiding  possible,  the  following 
integration  of  the  numerous  suggestions: 


the  types  and 
is  a  reasonable 


1 .  Alerting 

2.  Acquisition 

3.  Evaluation 

4.  Synthesis 

5.  Advising 


recognizing  that  a  particular  situation  now 
exists  or  warning  that  the  situation  is  likely 
to  arise 

identifying  information  sources,  selection 
among  sources,  and  information  retrieval  and 
display 

organizing  and  integrating  information, 
performing  calculations  including  predictions, 
and  identifying  model  parameters 

generating  alternatives  and  explaining  their 
origins,  strengths,  and  weaknesses 

monitoring  and  compensating  for  human  errors 
and  biases,  explaining  behavioral 
inconsistencies,  and  recommending  courses  of 
action . 


Aids  in  the  alerting  category  are  so  common  in  the  vehicle  and 
process  control  domains  that  one  seldom  thinks  of  them  as 
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decision  aids.  Most  current  aiding  systems  provide  support  in 
the  acquisition  and  evaluation  categories,  with  management 
information  systems  typically  focusing  on  the  acquisition  process 
while  decision  support  systems  usually  emphasize  evaluation. 

Many  of  the  more  recent  aiding  concepts  are  oriented  toward 
synthesis  and  advising.  Expert  systems  usually  are  designed  to 
generate  and  explain  alternatives  within  some  limited  problem 
domain.  These  systems  also  typically  provide  advice.  However, 
the  static  context-dominated  nature  of  most  current  expert 
systems  does  not  allow  for  the  adaptation  necessary  to  provide 
the  full  advisory  function  as  defined  above. 

Design  of  Decision  Aids 

The  literature  on  design  processes  deals  with  three  levels 
of  issue:  1)  system  design  in  general,  2)  design  of  decision 
aids,  and  3)  steps  of  decision  aiding.  The  nature  of  the 
literature  is  such  that  even  the  lowest  level  issues  are 
expressed  in  very  general  terms.  In  this  sense,  design  processes 
outline  an  ordered  set  of  issues  to  be  considered  rather  than 
specific  procedures  for  design. 

The  principles  and  guidelines  presented  in  the  literature 
provide  general  desiderata  and  specific  issues,  respectively,  to 
be  considered  in  designing  aids;  however,  unlike  design 


handbooks,  they  provide  little  direction  regarding  specific 
choices  in  addressing  issues  and  resolving  tradeoffs.  Many  of 
the  principles  and  guidelines  reflect  issues  relevant  to 
human-machine  interaction  in  general,  while  others  focus 
specifically  on  the  design  of  decision  aids.  As  might  be 
expected,  the  most  definitive  principles  and  guidelines  are  those 
that  have  been  culled  from  the  relatively  large  data  base  on 
human-machine  interaction  in  general.  Principles  and  guidelines 
specific  to  decision  aiding  tend  to  be  rather  broad  and  somewhat 
difficult  to  operationalize.  This  observation  will  be  elaborated 
upon  when  design  principles  and  guidelines  are  discussed  in  the 
next  section  in  terms  of  a  framework  for  adaptive  decision  aids. 


ADAPTIVE  DECISION  AIDS 


While  quite  a  few  documents  are  noted  above  as  dealing  with 
adaptive  decision  aids,  almost  all  of  these  documents  have  been 
generated  by  two  research  groups:  1)  Perceptronics ,  Inc.  and  2) 
the  authors'  group  while  at  the  University  of  Illinois  and,  more 
recently,  Georgia  Institute  of  Technology.  These  two  efforts  are 
briefly  reviewed  in  this  section  prior  to  introducing  a  general 
framework  for  adaptive  aiding. 


Background 


The  essence  of  Perceptronics'  approach  to  adaptive  aiding, 
at  least  initially,  has  been  the  use  of  multiattribute  utility 
functions  to  represent  decision  makers'  relative  preferences 
among  information  sources,  and  the  use  of  fairly  straightforward 
pattern  recognition  techniques  for  adapting  the  parameters  of  the 
utility  function  to  individual  decision  makers.  This  approach 
has  been  applied  to  computer-aided  information  selection  via 
utility  maximization  in  aircraft,  antisubmarine  warfare,  and 
command  and  control  environments.  Recently,  Perceptronics  has 
integrated  their  basic  approach  with  rule-based  models  and 
simulation  to  allow  application  of  their  adaptive  aiding  notions 
to  a  wider  range  of  decision  tasks. 


The  authors  and  their  colleagues  have  pursued  the  problem  of 


adaptively  (i.e.,  dynamically  in  time)  allocating  decision  making 
tasks  between  humans  and  computers.  Succinctly,  the  idea  has 
been  to  use  the  computer  to  make  the  allocation  decision  (or 
recommendation)  so  as  to  minimize  task  delay  time,  subject 
perhaps  to  maintaining  human  workload  at  a  satisfactory  level.  A 
queueing  theoretic  model  was  used  to  formulate  this  problem  and 
provide  the  basis  for  the  aiding  algorithm  which  was  tested  in 
the  aircraft  domain. 

The  feasibility  of  this  approach  to  task  allocation  depends 
on  the  resolution  of  several  issues  associated  with 
human-computer  communication.  Via  simulation  studies,  it  was 
found  that  computer  aiding  would  be  most  effective  if  the 
computer  knew  the  human's  current  state  of  knowledge,  current 
allocation  of  attention,  and  intended  allocation  of  attention. 
Because  of  the  cost  (i.e.,  time  and  workload)  of  having  the  human 
overtly  or  explicitly  communicate  this  type  of  information, 
covert  or  implicit  approaches  to  communication  were  considered 
including  direct  but  unobtrusive  observation  (e.g.,  eye 
tracking),  inference  via  physiological  measures,  and  inference 
via  model  matching.  Model  matching  has  been  the  approach  which 
has  received  most  attention  with  several  models  having  been 
developed  which  worked  quite  well  in  laboratory  evaluations. 

The  work  of  Perceptronics  and  that  of  the  authors  and  their 
colleagues  are  examples  of  situations  where  the  computer  adapts 
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itself  to  the  human.  An  alternative  is  to  have  the  human  adapt 
the  computer  to  himself.  Examples  in  the  literature  reviewed 
include  user  personalization  of  display  formats,  customization  of 
interface  features,  and  online  modification  of  outputs  of  aiding 
algorithms.  One  of  the  tangible  benefits  of  the  user-initiated 
approach  is  that  the  human  then  clearly  knows  what  adaptation  has 
occurred.  In  contrast,  if  the  computer  automatically  adapts  to 
the  user,  it  can  prove  difficult  to  supply  the  user  with 
meaningful  and  easy-to-digest  information  regarding  the  status  of 
the  adaptive  system  (e.g.,  who  has  responsibility  for  which 
tasks) . 

While  much  of  the  research  reviewed  thus  far  in  this  section 
has  addressed  some  of  the  central  questions  in  adaptive  aiding, 
this  overall  set  of  efforts  has  been  far  from  comprehensive. 
What  has  been  lacking  is  a  framework  within  which  to  view  the 
general  topic  of  adaptive  decision  aids.  The  remainder  of  this 
section  is  devoted  to  proposing  such  a  framework. 

Overall  Framework 

Conceptual  frameworks  should  be  developed  to  serve 
particular  purposes.  In  that  sense,  no  useful  framework  is 
completely  general.  However,  it  is  possible  to  generalize  in  the 
sense  of  serving  a  specific  purpose  across  multiple  problem 
domains.  For  this  report,  the  goal  is  to  suggest  a  framework 


that  outlines  the  research  issues  in  adaptive  aiding  in  a  manner 
that  is  relevant  to  vehicle  and  process  systems,  command  and 
control,  management  systems,  etc. 

The  overall  structure  of  the  proposed  framework  involves  the 
following  six  attributes  of  an  adaptive  decision  aid: 

1.  Type  of  decision  task 

2.  Level  of  decision  aiding 

3-  Form  of  adaptation 

4.  Mode  of  adaptation 

5.  Method  of  adaptation 

6.  Means  of  communication 

The  first  two  attributes  (i.e.,  type  of  task  and  level  of  aiding) 
were  discussed  in  the  previous  section  and  need  little  further 
elaboration  at  this  point  except  perhaps  to  point  out  that 
resolution  of  the  issues  associated  with  the  other  four 
attributes  (i.e.,  form,  mode,  method,  and  means)  is  often  likely 
to  depend  on  the  type  of  task  and  level  of  aiding. 


The  form  of  adaptation  relates  to  the  question: 


what  is 


adapted  to?  This  question  can  be  answered  at  several  levels.  At 
the  highest  level,  adaptation  can  be  relative  to  the  task  or 
user.  At  the  lower  levels  of  detail,  adaptation  can  involve  a 
class  of  tasks  or  users,  a  particular  member  of  a  class,  or  the 
state  of  a  particular  member.  For  example,  an  aid  might  adapt  to 
the  class  of  all  impulsive  or  field  dependent  decision  makers,  or 
to  a  particular  decision  maker  who  exibits  some  specific  degree 
of  membership  in  the  class,  or  to  a  decision  maker  in  a 
particular  state  at  a  specific  time  (e.g.,  high  workload  two 
minutes  prior  to  touchdown). 


Therefore,  adaptation  can  be  relative  to  the  task  or  user 
and,  within  these  categories,  relative  to  a  class,  member,  or 
state.  Virtually  any  design  is  adapted  (albeit  offline  by  the 
designer)  to  a  class  of  tasks  or  users.  Thus,  the  key 
distinction  here  is  between  adapting  to  members  (e.g.,  particular 
users)  versus  states  (e.g.,  specific  situations  at  particular 
points  in  time). 


Mode  of  Adaptation 

The  mode  of  adaptation  concerns  the  question:  who  does  the 
adapting?  There  are  three  possible  answers  to  this  question:  l) 
designer ,  2)  user,  and  3)  the  aid  itself.  A  further  aspect  of 
the  mode  of  adaptation  is  whether  it  is  done  offline  or  online . 
Any  adaptation  by  the  designer  is,  by  definition,  performed 


offline  while  users  and  aids  usually  adapt  online.  However,  it 
is  easy  to  envision  users,  for  example,  personalizing  display 
formats  and  customizing  interface  features  in  an  offline  mode. 
Of  course,  in  this  mode,  the  distinction  between  designer  and 
user  becomes  fuzzy  unless  one  specifies  that  designers' 
adaptations  are,  by  definition,  always  for  other  than  themselves. 

Designer-offline  adaptation  typically  involves  classes  of 
tasks  and  users.  This  combination  of  the  attributes  of  "form" 
and  "mode"  is  practically  identical  to  how  one  might  characterize 
the  design  of  nonadaptive  aids.  From  this  perspective,  one  can 
view  more  traditional  decision  aids  as  special  cases  of  adaptive 
aids  where  all  adaptation  is  done  offline  by  a  designer  who 
attempts  to  anticipate  task  and  user  characteristics. 

Method  of  Adaptation 

The  method  of  adaptation  relates  to  the  question:  how  is 
the  adaptation  done?  In  general,  adaptation  involves 
measurements  which  are  manipulated  using  models  such  that 
modification  of  various  types  may  result.  A  continuum  of 
modification  is  possible.  The  highest  level  modification  is  the 
allocation  of  tasks  (e.g.,  who  does  the  information  selection). 
A  lower  level  modification  involves  partitioning  of  a  task  (e.g., 
human  chooses  setpoints  and  computer  controls  relative  to  this 
reference).  The  lowest  level  modification  involves 


transformation  of  a  task  (e.g.,  abstracting  a  display  from  a 
physical  to  a  functional  representation). 


!>■ 


The  range  of  measurements  and  models  involved  with  these 
levels  of  modification  can  be  rather  large.  The  particular 
choices  among  alternative  measures  and  models  are  dictated  by  the 
level  of  modification  desired  as  well  as  the  form  and  mode  of 
adaptation,  with  of  course  consideration  given  to  technical 
feasibility  and  reliability.  Typically,  measures  are  difficult 
to  obtain,  while  there  are  a  plethora  of  reasonable  models,  among 
which  the  choice  is  highly  influenced  by  the  modeler's 
experiences  and  preferences. 


Alternative  types  of  measurement  include  task  variables 
(e.g.,  the  state  of  the  system),  user  variables  (e.g.,  workload), 
user  task-related  responses  (e.g.,  usage  of  commands),  and  user 
adaptation-related  specifications  (e.g.,  format  and  dialogue 
preferences).  Types  of  model  can  range  from  quantitative 
computational  models  that  allow  prediction  of  overall  performance 
(e.g.,  manual  control  models),  to  models  that  predict  particular 
choices  (e.g.,  decision  theoretic  models),  to  qualitative  models 
that  are  concerned  with  predicting  modes  of  behavior  (e.g., 
pattern  recognition  vs.  structural  search)  instead  of  particular 
responses . 
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Means  of  Communication 


The  means  of  communication  refers  to  the  manner  in  which 
measurements  are  made  and  the  way  in  which  users  and  aids  inform 
each  other  of  the  status  of  the  decision  making  process.  As 
noted  earlier  in  this  section,  communication  can  be  either 
explicit  (overt)  or  implicit  (covert).  Explicit  communication 
may  involve  specific  displays  and  controls,  structured  dialogues 
via  keyboards  or  voice,  or  natural  language  via  keyboards  or 
voice.  Implicit  communication  can  be  accomplished  using 
unobtrusive  but  direct  observations,  indirect  measurements,  or 
inference. 


The  distinction  between  explicit  and  implicit  communication 
relates  to  the  user's  awareness  of  the  communication.  Therefore, 
explicit  communication  is  such  that  the  user  is  aware  that  he  or 
she  is  communicating  with  the  aid.  In  contrast,  implicit 
communication  is  such  that  the  user  will  only  be  aware  of  doing 
the  task;  the  user  is  not  consciously  aware  of  the  fact  that  the 
aid  is  gleaning  information  from  these  task-related  activities. 


The  differentiation  of  direct,  indirect,  and  inferred 
measures  is  made  to  emphasize  the  degree  to  which  measurements 
may  have  to  be  transformed  and  interpreted.  A  direct  measurement 
of,  for  example,  aircraft  altitude  and  attitude  or  a  user's 
choice  between  modes  of  operation  requires  no  transformation  or 


interpretation; 


the  measurement  directly  supplies  the 
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information  sought.  Indirect  measurements  require 
transformations  (e.g.,  via  deductive  principles  of  physical  or 
behavioral  science)  in  order  to  obtain  the  desired  information. 
Inferred  measurements  require  interpretation  (e.g.,  induction  via 
models),  perhaps  to  the  extent  that  one  must  extrapolate  to 
obtain  the  information  of  interest. 

Summary  and  Example 

Figure  1  summarizes  the  form,  mode,  method,  and  means 
attributes  of  adaptive  decision  aids.  These  attributes  typically 
interact  with  the  type  of  task  and  level  of  aiding  attributes. 
However,  to  streamline  the  presentation,  these  two  attributes  are 
not  indicated  in  this  figure. 

Considering  the  various  types  of  task,  levels  of  aiding, 
forms,  modes,  methods,  and  means  in  Figure  1,  there  would  appear 
to  be  thousands  of  potential  adaptive  aids.  However,  th^ 
apparent  uniformity  and  completeness  of  the  classifications  shown 
in  this  figure  mainly  serve  to  clarify  the  presentation;  many  of 
the  combinations  indicated  are  neither  interesting  nor 
meaningful.  (See  notes  on  Figure.)  This  is  due  to  the  fact  that 
the  purpose  of  this  figure  is  not  enumeration  of  all  possible 
aids;  instead  its  purpose  is  to  clearly  summarize  the  issues  to 
be  addressed,  questions  to  be  answered,  and  the  nature  of 
possible  answers.  The  usefulness  of  such  a  framework  is 
illustrated  in  the  next  section  by  focusing  on  command  and 
control  systems. 
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FORM  OF  ADAPTATION:  WHAT  IS  ADAPTED  TO? 


CLASS 

MEMBER 

STATE 

USER 

1 

2 

2 

TASK 

1 

2 

i 

2 

DESIGNER 

USER 

AID 

OFFLINE 

1 

2 

3 

ONLINE 

4 

3 

2 

METHOD  OF  ADAPTATION:  HOW  IS  THE  ADAPTATION  DONE? 

ALLOCATING 

PARTITIONING 

TRANSFORMATION 

MEASUREMENTS 

2 

2 

) 

2 

MODELS 

2 

2 

2 

MEANS  OF  COMMUNICATION:  HOW  IS 

INFORMATION  TRANSMITTED? 

j  DIRECT 

|  INDIRECT 

INFERRED 

EXPLICIT 

IMPLICIT 

Notes: 


1  =  always  combined 

2  =  frequently  combined 

3  =  seldom  combined 

4  =  never  combined 


2 

3 


3 

2 


3 

2 


Figure  1 .  Four  Attributes  of  Adaptive  Decision  Aids 
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However,  rather  than  immediately  consider  a  domain  as 
complex  as  command  and  control,  the  use  of  the  framework 


summarized  in  Figure  1  can  be  illustrated  with  a  more 
straightforward  example.  Consider  the  task  of  computer-aided 
pattern  recognition  where  the  patterns  of  interest  are  such  that 
human  abilities  surpass  those  of  the  computer  unless  the  time 
pressure  on  the  human  becomes  excessive;  then,  the  computer's 
abilities  are  better  and  acceptable.  The  design  problem  is  to 
develop  a  mechanism  for  determining  who  (human  or  computer) 
should  perform  the  task  at  any  point  in  time. 

The  degradation  of  human  pattern  recognition  abilities  with 
increasing  time  pressure  is  likely  to  depend  on  the  complexity  of 
the  patterns  being  viewed.  Recognition  performance  for  simple 
patterns  (in  simple  backgrounds)  is  likely  to  be  maintained  for 
all  but  extreme  levels  of  time  pressure.  However,  for  complex 
patterns  (in  complex  backgrounds),  recognition  performance  is 
likely  to  be  quite  sensitive  to  time  pressure.  Therefore,  it  is 
probably  reasonable  to  allocate  the  recognition  task  on  the  basis 
of  both  pattern  complexity  and  time  pressure. 

Figure  1  can  be  used  as  a  basis  for  exploring  alternative 
approaches  to  this  design  problem.  Considering  the  form  of 
adaptation  (i.e.,  what  is  adapted  to?),  any  viable  aid  would  have 
to  be  designed  to  at  least  be  consistent  with  a  class  of  patterns 
(e.g.,  aerial  reconnaissance)  and  a  class  of  users  (e.g., 
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aircraft  pilots).  As  noted  earlier,  this  would  represent  a 
baseline  nonadaptive  aid. 

If  the  aiding  mechanism  was  adapted  (e.g.,  via  threshold 
parameters)  relative  to  particular  types  of  pattern  (e.g.,  enemy 
tanks  in  wooded  background)  and/or  specific  levels  of  time 
pressure  (e.g.,  nap-of-the-earth  flight  at  a  given  speed),  then 
the  form  of  adaptation  would  be  "task-member."  If  pattern  or 
background  characteristics,  or  time  pressure  parameters,  were 
measured  and  updated  online  with  subsequent  adaptations  of  the 
aiding  mechanism,  the  form  would  be  "task-state."  Considering  the 
user,  if  an  individual's  score  on  a  pattern  recognition  test 
(e.g.,  the  embedded  figures  test)  were  used  to  adapt  aiding 
parameters,  the  form  would  be  "user-member."  Finally,  if 
adaptation  were  on  the  basis  of  an  online  measurement  of  mental 
workload,  the  form  of  adaptation  would  be  "user-state."  From  this 
example,  it  should  be  clear  that  a  particular  aid  can  embody  more 
than  one  form  of  adaptation  (e.g.,  "task-member"  and 
"user-state" ) . 

With  regard  to  the  mode  of  adaptation  (i.e.,  who  does  the 
adapting?),  the  baseline  nonadaptive  aid  would  be  such  that  the 
designer  would  adjust  the  aiding  mechanism  offline  to 
"task-class"  and  "user-class."  If  the  aid  adapted  its  mechanism 
prior  to  task  performance  relative  to  expected  pattern  and 
background  characteristics,  as  well  as  the  anticipated  level  of 


time  pressure,  the  mode  of  adaptation  would  be  "aid-offline."  If 
these  variables  were  assessed  and  adapted  to  online  by  the  aid, 
the  mode  would  obviously  be  "aid-online."  If  the  user  adapted  the 
parameters  of  the  aiding  mechanism  relative  to  these  variables 
prior  to  or  during  task  performance,  the  mode  would  be 
"user-offline"  or  "user-online,"  respectively. 


Considering  the  method  of  adaptation  (i.e.,  how  is  the 
adaptation  done?),  the  pattern  recognition  task  could  be 
allocated  to  either  human  or  computer.  Alternatively,  the  task 
could  be  partitioned  into  subtasks  (e.g.,  human  identifies 
feasible  set,  computer  evaluates  degree  of  match  of  each 
candidate,  and  human  chooses  among  evaluated  alternatives). 
Aided  transformation  might  involve  smoothing,  rotating,  or 
inverting  the  displayed  patterns. 


The  choice  among  these  types  of  modification  tfpvld  depend  on 
the  task  characteristics  of  pattern  complexity  and  level  of  time 
pressure  as  well  as  user  characteristics  such  as  individual 
pattern  recognition  abilities,  resilience  to  time  pressure,  and 
perhaps  current  workload.  Some  of  these  variables  could  be 
measured  directly;  others  could  be  deduced  from  indirect 
measurements;  and  some  could  be  inferred  using  models  of  human 
performance.  These  measurements  would  serve  as  inputs  to  a  model 
that  would  predict  the  most  appropriate  level  of  modification 
(i.e.,  allocated,  partitioned,  or  transformed).  This  model  might 


be  simply  a  look-up  table  of  precomputed  aiding  parameters  as  a 
function  of  task  and  user  variables.  Or,  the  model  might  be  more 
computational  in  manner,  perhaps  updating  deductions  and 
inferences  as  measurements  change. 

With  regard  to  the  means  of  communication  (i.e.,  how  is 
information  transmitted? ) ,  any  direct  communications  would  depend 
on  explcit  user  involvement.  For  example,  the  user  might  specify 
pattern  complexity.  However,  indirect  and  inferred 
communications  are  more  likely.  An  example  of  indirect 
communication  would  be  the  transformation  of  altitude  and 
airspeed  into  an  estimated  viewing  time  for  target  patterns.  If 
the  user  specifies  altitude  and  airspeed,  the  means  would  be 
"explicit-indirect;"  if  the  measurement  and  transformation 
proceeds  without  user  involvement,  the  means  would  be 
"implicit-indirect."  Another  example  of  indirect  communication 
would  be  user  specification  of  his  ID  number  which  would  enable 
access  to  a  data  base  of  characteristics  for  the  particular  user. 
Since  the  user  is  likely  to  be  aware  of  why  he  is  supplying  the 
ID  number,  the  means  would  be  "explicit-indirect." 

Inferred  communications  are  usually  implicit,  unless  the  aid 
explicitly  communicates  with  the  user  to  confirm  its  inferences. 
An  example  of  model-based  inference  would  be  the  use  of 
performance  on  a  concurrent  task  (e.g.,  rms  control  activity)  and 
an  indirect  assessment  of  time  pressure  (e.g.,  via  altitude  and 
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airspeed)  as  inputs  to  a  model  that  would  enable  one  to  infer  the 
level  of  complexity  of  target  patterns.  For  a  task  such  as 
time-pressured  pattern  recognition,  it  is  likely  that  most 
communications  involving  measurements  of  variables  would  be 
"implicit-indirect"  or  "implicit-inf erred . " 

Another  aspect  of  communication  involves  feedback  to  the 
user  regarding  the  current  level  of  modification.  If  the 
modification  involves  allocation,  communication  obviously  must  be 
"explicit-direct."  A  modification  involving  partitioning  would 
probably  be  "explicit-direct,"  although  one  can  imagine 
situations  where  the  user  would  only  be  implicitly  aware  that  the 
aid  was  performing  an  unseen  portion  of  the  task.  Transformation 
is  a  type  of  modification  that  might  very  well  be  communicated 
implicitly  and  perhaps  involve  indirect  measurements  or  inference 
on  the  part  of  the  user. 

This  brief  example  has  illustrated  how  the  framework 
summarized  in  Figure  1  can  be  used  to  pose  design  questions  and 
the  nature  of  answers.  While  it  is  very  difficult,  if  not 
impossible,  to  enumerate  alternative  answers  to  each  type  of 
question  (particularly  for  measurements  and  models),  the 
framework  helps  the  designer  to  consider  systematically  the  range 
of  issues  of  importance.  The  following  section  pursues  this 
approach  for  identifying  adaptive  aiding  issues  in  command  and 


control . 


IMPLICATIONS  FOR  COMMAND  AND  CONTROL 


Several  documents  (i.e.,  8)  provide  overviews  of  command  and 
control  within  particular  military  domains.  The  functions  or 
missions  discussed  include:  1)  aircraft  operations; 
interdiction,  close  air  support,  interception,  reconnaissance, 
defense  suppression,  and  air  lift,  2)  naval  operations; 
antisubmarine,  antisurface,  and  antiairwarfare ,  and  3)  other; 
amphibious  assault  and  surface-to-surface  missile  warfare.  This 
great  variety  of  functions  makes  it  very  difficult  to  formulate  a 
general  framework  for  command  and  control.  However,  if  one 
considers  the  decision  making  tasks  involved  in  accomplishing 
these  functions,  a  fairly  general  structure  emerges. 


Figure  2  is  a  synthesis  of  the  various  views  of  command  and 
control  decision  making  expressed  in  the  literature  reviewed.* 
This  figure  is  basically  an  elaboration  of  earlier  discussions  in 
this  report,  with  emphasis  on  common  attributes  of  tasks  as  well 
as  relationships  among  tasks.  The  first  portions  of  this  section 
discuss  the  nature  of  the  tasks  depicted  in  Figure  2.  Following 
this  discussion,  possible  aids  for  these  tasks  are  considered. 


*It  should  be  emphasized  that  this  synthesis  is  based  almost 
totally  on  the  authors'  previous  efforts  in  decision  making 
research  as  well  as  conceptual  frameworks  proposed  by  other 
authors.  It  is  interesting  to  note  how  a  number  of  researchers 
have  all  advocated  basically  the  same  conceptualization,  but  with 
different  terminology.  Certainly  the  synthesis  presented  here 
is.  in  essence,  another  variation  of  this  standard  theme. 
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SELECTION  AMONG  ALTERNATIVE  INFORMATION  SOURCES 


GENERATION  OF  ALTERNATIVE  EXPLANATIONS 
EVALUATION  OF  ALTERNATIVE  EXPLANATIONS 
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EVALUATION  OF  ALTERNATIVE  COURSES  OF  ACTION 
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IMPLEMENTATION  OF  PLAN /OBSERVATION  OF  CONSEQUENCES 
EVALUATION  OF  DEVIATIONS  FROM  EXPECTATIONS 
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Figure  2.  Command  and  Control  Decision  Making  Tasks 


Situation  Assessment:  Information  Seekin, 


Situation  assessment  can  be  viewed  as  involving  two  phases: 
information  seeking  and  explanation.  Information  seeking 
includes  generating/ identifying ,  evaluating,  and  selecting  among 
alternative  information  sources.  The  alternative  information 
sources  are  usually  fairly  well  defined  in  command  and  control. 
They  include  data  bases,  sensors,  and  models  which  can  be  viewed 
as  providing  information  about  what  has  happened  (past),  is 
happening  (present),  and  may  happen  (future),  respectively. 

Generation/identification  of  alternative  information  sources 
involves  rapidly  (and  perhaps  unconsciously)  considering  the 
large  number  of  information  sources  available  and  delimiting  a 
reasonable  subset  for  further  consideration.  Evaluation  of 
information  sources  involves  assessing  their  relevance, 
information  content,  and  resource  requirements. 

Selection  among  alternative  information  sources  basically 
involves  the  allocation  of  information  acquisition  resources 
(e.g.,  people,  sensors,  and  bandwidth)  relative  to  criteria  such 
as  uncertainty  reduction  and  resource  constraints.  Given  the 
outputs  of  the  evaluation  task  and  clear  statements  of  the 
allocation  criteria,  selection  in  terms  of  resource  allocation 
can  be  posed  as  a  rather  standard  optimization  problem.  However, 
humans  do  not  appear  to  approach  selection  quite  so  rigorously 


which  may  provide  opportunities  for  aiding  as  is  discussed  later. 

Situation  Assessment :  Explanation 

The  explanation  phase  of  situation  assessment  includes 
generating,  evaluating,  and  selecting  among  alternative 
explanations  of  the  situation.  Generating  alternatives  involves 
synthesizing  possible  explanations  of  what  has  happened,  is 
happening,  and  may  happen.  Typically,  there  is  a  default 
explanation  which  concludes  that  a  priori  expectations  have  been, 
are,  or  will  be  fulfilled.  The  process  whereby  other 
alternatives  emerge  appears  to  depend  on  previous  experiences, 
pattern  recognition  abilities,  and  perhaps  creativity. 

Evaluation  of  alternatives  involves  finding  the  degree  of 
correspondence  between  each  candidate  explanation  and  the 
assessed  situation.  Beyond  assessing  the  degree  of  fit, 
evaluation  also  includes  determining  the  cost  of  both  types  of 
misevaluation  (i.e.,  false  acceptance  and  rejection  of 
alternatives).  In  general,  the  cost  of  falsely  accepting  an 
alternative  depends  on  the  particular  alternative  that  is 
consequently  falsely  rejected.  In  ambiguous  situations  where 
many  alternative  explanations  are  feasible,  the  interdependencies 
of  these  costs  of  error  can  be  complicated  to  keep  in  mind  and 
would  seem  to  be  a  possible  area  for  computer  aiding. 


Selection  among  alternatives  involves  trading  off  the 
feasible  explanations  in  terms  of  the  degree  of  correspondence 
with  the  assessed  situation  and  the  costs  of  false  acceptance  and 
rejection.  If  the  values  of  all  of  the  parameters  associated 
with  these  tradeoffs  were  known,  selection  could  be  viewed  as  a 
minimum  expected  cost  optimization  problem.  However,  it  is  very 
unlikely  that  humans  make  selections  this  formally,  partially 
because  of  inherent  ambiguities  in  situations  and  explanations, 
and  partially  due  to  human  information  processing  limitations. 
Therefore,  as  with  selection  among  information  sources,  selection 
among  explanations  may  be  an  appropriate  task  for  aiding. 

Planning  and  Commitment 

Planning  and  commitment  includes  generating,  evaluating,  and 
selecting  among  alternative  courses  of  action.  Planning  differs 
from  situation  assessment  primarily  in  terms  of  being  purely 
future-oriented  and  emphasizing  a  sequence  of  actions  over  time 
rather  than  an  assessment  at  a  particular  point  in  time.  Thus, 
while  situation  assessment  attempts  to  determine  what  has 
happened,  is  happening,  or  may  happen,  planning  focuses  on 
manipulating  future  situations  by  choosing  appropriate  courses  of 
action. 

Generating  alternative  courses  of  action  is  similar  to 
generating  alternative  explanations.  Typically,  courses  of 
action  that  have  been  successfully  used  before  in  the  given 


situation  will  be  considered.  Often,  such  courses  of  action  or 
plans  may  be  available  in  the  form  of  procedures.  Alternative 
plans  also  may  emerge  from  experiences  with  analogous  situations 
or,  if  absolutely  necessary,  via  analytical  study  of  the 
situation.  It  is  difficult  to  say  where  truly  novel  alternatives 
come  from;  while  this  makes  alternative  generation  a  difficult 
process  to  aid,  it  nevertheless  does  support  a  design  principle 
requiring  sufficient  system  flexibility  to  avoid  inhibiting 
creativity  when  it  is  needed. 

Evaluation  of  alternative  courses  of  action  involves 
assessing  or  imagining  the  consequences  of  plans,  both  in  terms 
of  impact  on  future  situations  and  resource  requirements.  This 
type  of  evaluation  is  usually  performed  by  mapping  (correctly  or 
otherwise)  from  possible  actions  to  previously  experienced 
consequences.  In  many  situations,  training  will  enable 
reasonably  accurate  anticipation  of  effects  of  actions.  When 
costs  of  forecasting  errors  are  high  and  time  allows,  evaluation 
of  alternative  actions  may  be  performed  using  predictive  models, 
perhaps  the  simplest  operational  form  of  which  is  a  predictor 
display. 

Selection  among  alternative  courses  of  action  involves  the 
allocation  of  action  resources  (e.g.,  people  and  equipment) 
relative  to  criteria  which  assess  the  degree  to  which  objectives 
are  achieved  over  some  planning  horizon,  and  subject  to  resource 


constraints.  As  with  the  other  types  of  selection  task,  given 
sufficient  information,  selection  among  alternative  courses  of 
action  can  be  posed  as  a  constrained  optimization  problem. 
However,  the  complexity  of  such  a  formulation  dictates  that 
unaided  humans  are  very  unlikely  to  pursue  selection  in  this 
manner.  While  aiding  may  be  possible,  its  feasibility  is  likely 
to  depend  totally  on  the  availability  of  types  of  information 
which  humans  may  have  difficulty  assessing  and/or  communicating. 

Execution  and  Monitoring 

Execution  and  monitoring  include  implementing  the  plan 
selected,  observing  its  consequences,  evaluating  deviations  of 
observed  consequences  from  expectations,  and  selecting  between 
accepting  and  rejecting  deviations  as  being  within  the  range  of 
possibilities  given  chance  variations  in  consequences.  Plan 
implementation  involves  control  in  the  sense  of  communicating  and 
manipulating,  and  coordination  in  terms  of  sequencing  and 
balancing  the  deployment  of  resources.  Observation  of 
consequences  involves  acquiring  information  and  correlating  it 
with  the  course  of  action  implemented. 

Evaluation  of  deviations  of  observed  consequences  from 
expectations  is  the  process  of  determining  whether  or  not  the 
actual  course  of  events  significantly  deviates  from  what  was 
anticipated  to  be  the  result  of  plan  implementation.  While  some 
deviations  are  quite  normal,  excessive  deviations  indicate  that 


something  is  awry  (e.g.,  the  situation  assessment  may  have  been 
wrong).  Obviously,  the  quantitative  definition  of  "excessive” 
depends  on  the  uncertainty  associated  with  a  priori  expectations 
of  consequences. 

Selection  between  acceptance  and  rejection  of  the  observed 
deviations  as  normal  involves  trading  off  the  costs  of  false 
acceptance  or  rejection  in  terms  of  the  costs  of  incorrectly 
proceeding  with  execution  and  the  costs  of  wrongly  abandoning 
execution  to  return  to  situation  assessment  and/or  planning. 
Given  sufficient  information,  this  decision  task  can  be  viewed  as 
simply  one  of  choosing  the  minimum  expected  cost  alternative. 
However,  since  probabilities  and  costs  of  selection  errors  are 
usually  not  explicit,  this  selection  task,  as  with  the  selection 
tasks  discussed  earlier,  is  not  so  straightforward. 

Input/ Output 

The  terms  used  in  Figure  2  and  the  above  discussion  were 
designed  to  emphasize  common  attributes  of  decision  making  tasks 
in  command  and  control.  Clearly,  the  three  most  central  words  in 
this  formulation  are  generation.  evaluation,  and  selection 
relative  to  alternative  information  sources,  explanations, 
courses  of  action,  and  deviations  from  expectations.  Beyond 
these  types  of  decision,  there  are  input  and  output  tasks  that 
are  common  to  all  of  the  decision  making  tasks.  Since  the  same 
information  displays,  input  devices,  and  dialogue  structure  can 
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support  different  decision  making  tasks,  albeit  perhaps  in 
different  ways,  these  input/output  tasks  are  not  shewn  in  Figure 
2.  Therefore,  for  example,  the  act  of  obtaining  information  from 
displays  is  not  viewed  as  a  separate  decision  making  task  per  se. 

Nevertheless,  these  input/output  activities  are  good 
candidates  for  aiding  in  order  to  free  humans  to  attend  to 
generation,  evaluation,  and  selection.  For  example,  acquisition 
and  integration  of  information  is  an  area  where  aiding  can  be 
quite  useful.  Acquisition  and  integration  of  information 
involves  querying  data  bases,  sensors,  and  models,  and 
integrating  the  information  produced  into  a  coherent  picture. 
The  acquisition  process  is  fairly  straightforward  if  controls  and 
displays  are  reasonably  designed.  Integration  is  more  subtle  in 
that  it  usually  relies  on  humans'  impressive  pattern  recognition 
abilities.  These  abilities  can  be  exploited  by  appropriate 
transformation,  formatting,  and  coding  of  information  to  match 
individual  differences  as  well  as  population  stereotypes. 

Opportunities  for  Aiding 

The  discussion  has  now  progressed  to  the  point  that  aiding 
can  be  considered  in  terms  of  five  generic  tasks:  generation, 
evaluation,  selection,  input,  and  output.  From  a  human 


information  processing  point  of  view,  these  five  tasks  represent 
three  types  of  processing:  1)  input  or  perceptual  processing,  2) 
central  or  cognitive  processing  (i.e.,  generation,  evaluation, 


I 

and  selection),  and  3)  output  processing.  Thus,  these  tasks  and 
the  alternative  aiding  concepts  discussed  in  this  and  the 
following  subsection  cover  the  full  range  of  human  information 
processing . 

In  order  to  suggest  potential  aids,  assumptions  must  be  made 
regarding  the  availability  of  information  to  the  aid.  There  are 
basically  three  sources  of  information.  The  most  obvious  is 
measurements  which  are,  for  example,  the  typical  approach  for 
assessing  system  state  variables,  (i.e.,  velocities,  altitudes, 
temperatures,  pressures,  etc.).  A  second  source  of  information 
is  user  specifications  where,  for  instance,  the  user  might 
specify  final  decisions  regarding  the  situation  assessed, 
explanation  accepted,  or  plan  chosen.  The  third  source  of 
information  is  inferences  based  on  measurements  and 
specifications . 

In  the  following  discussion,  the  phrase  "For  a  given...,"  is 
frequently  used.  The  word  "given"  is  defined  here  as  the  product 
of  accessing  information  via  direct  or  indirect  measurements, 
user  specifications,  and/or  inferences.  Thus,  the  word  "given" 
is  used  here  to  state  the  a  priori  knowledge  an  aid  is  assumed  to 
have . 


Aids  for  generation  are  among  the  most  difficult  to  imagine 
since  generation  can  be  a  rather  creative  activity.  However, 
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while  creativity  may  be  difficult  to  replace,  it  can  be 
supported.  The  following  levels  of  aiding  seem  feasible: 


1.  For  a  given  situation,  an  aid  might  retrieve  previously 
relevant  and  useful  alternatives. 

2.  For  a  given  set  of  attributes,  an  aid  might  retrieve 
candidate  alternatives  with  these  attributes. 


3.  Given  feedback  with  regard  to  suggested  alternatives,  an  aid 
might  adapt  its  search  strategy  and/or  tactics. 


Aids  for  evaluation  are  among  the  more  straightforward.  To 
a  great  extent,  such  aids  are  basically  information  retrieval  and 
calculation  routines.  While  the  human  factors  issues  associated 
with  specifying  and  displaying  this  information  are  by  no  means 
trivial,  these  issues  are  related  to  input  and  output  rather  than 
evaluation.  Possible  types  of  aiding  for  evaluation  include: 


1  .  For  a  given  alternative,  an  aid  might  assess  the 

alternative's  a  priori  characteristics  such  as  relevance, 
information  content,  and  resource  requirements. 

2.  For  a  given  situation  and  alternative,  an  aid  might  assess 

the  degree  of  correspondence  between  situation  and 
alternative . 

3.  For  a  given  alternative,  an  aid  might  assess  the  likely 

future  consequences  such  as  expected  impact  and  resource 
requirements  if  the  alternative  is  chosen. 

4.  For  given  multiple  alternatives,  an  aid  might  assess  the 

merits  of  each  alternative  relative  to  the  other 

alternatives . 
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5.  Given  feedback  of  appropriate  variables,  an  aid  might  adapt 
its  evaluations  in  terms  of  time  horizon,  accuracy,  etc. 


As  noted  in  earlier  discussions,  selection  can  be  posed  as 
an  optimal  resource  allocation  problem  if  sufficient  information 
is  available.  While  information  is  often  lacking,  efficient 
methods  (i.e.,  minimum  number  of  questions)  of  querying  the 
decision  maker  have  recently  been  developed  and  would  seem  to 
offer  a  means  of  ameliorating  this  difficulty.  Potential  types 
of  aiding  for  selection  include: 


1.  For  given  criteria  and  set  of  evaluated  alternatives,  an  aid 
might  suggest  the  selection  that  yields  the  best  (e.g., 
minimum  cost)  allocation  of  information  acquisition  and/or 
action  resources  relative  to  given  resource  constraints  and 
planning  horizon. 

2.  For  given  individual  differences  and  time-variations  of 
criteria,  preferences,  and  evaluations,  an  aid  might  adapt 
its  suggestions  to  reflect  these  variations. 


Input  tasks  are  concerned  with  acquisition  and  integration 
of  information  as  well  as  observation  of  consequences.  Aids  for 
input  tasks  basically  involve  the  way  in  which  information  is 
shown  to  the  user.  Possible  aids  include: 


1.  For  given  information,  an  aid  might  transform,  format,  and 
code  the  information  to  avoid  human  limitations  snd  exploit 
human  capabilities. 


3.  For  a  given  sample  of  information,  an  aid  might  fit  models  to 
the  information  in  order  to  integrate  and  interpolate  within 
the  sample. 


4.  For  given  constraints  (e.g.,  time)  and  individual 
differences,  an  aid  might  adapt  transformations,  models,  etc. 
accordingly . 


Output  tasks  are  those  associated  with  plan  implementation. 
While  such  tasks  are  not  usually  classified  as  decision  making, 
aiding  of  implementation  can  be  a  very  important  aspect  of  a 
decision  support  system.  The  following  levels  of  aiding  appear 
feasible: 


1.  For  a  given  plan  and  information  regarding  the  user's 
actions,  an  aid  might  monitor  implementation  for 
inconsistencies  and  errors  of  omission  and  comission. 


2.  For  a  given  plan  and  information  regarding  the  user's 

actions,  an  aid  might  perform  some  or  all  of  the 

implementation  to  compensate  for  the  user's  inconsistencies, 
errors,  or  lack  of  resources  (e.g.,  time). 

3.  Given  information  on  workload,  urgency,  etc.,  an  aid  might 
adapt  its  monitoring  and/or  implementation  accordingly. 


The  various  opportunities  for  aiding  discussed  above  are  not 
necessarily  all  of  the  possibilities.  However,  they  are 
representative  of  what  has  been  tried  or  at  least  suggested  in 
the  literature.  Given  these  alternatives,  the  next  topic  of 


discussion  is  the  possibilities  for  adaptive  aiding. 
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Possibilities  for  Adaptive  Aidin 


Starting  with  the  above  outline  of  aiding  opportunities,  the 
framework  for  adaptive  aiding  introduced  earlier  can  be  used  to 
define  and  explore  the  issues  associated  with  adaptive  aids.  As 
noted  earlier,  the  main  purpose  of  this  framework  is  to  provide  a 
structured  set  of  questions,  as  well  as  possible  categories  of 
answers,  that  should  be  addressed  when  designing  adaptive  aids. 
From  this  perspective,  it  is  quite  unnecessary  to  attempt  to 
provide  entries  for  each  element  of  each  tabulation  in  Figure  1 . 
Instead,  as  is  illustrated  in  this  section,  Figure  1  should  be 
used  to  motivate  and  organize  exploration  of  relevant  issues. 


To  illustrate  this  process,  adaptive  aids  for  generation  of 
alternatives  are  first  considered.  The  initial  question  in 
Figure  1  is:  "What  is  adapted  to?"  An  appropriate  answer  to  this 
question  is  that  the  aid  for  retrieving  alternatives  might  adapt 
its  search  strategy  to  the  situation,  desired  attributes  of 
alternatives,  and/or  user  feedback  in  the  sense  of  judgements  of 
the  relevance  of  initially  retrieved  alternatives.  Using  the 
terminology  in  Figure  1 ,  these  three  types  of  adaptation  involve 
members  and  states  of  the  task  dimension  and  states  of  the  user 
dimension,  respectively. 


The  second  question  in  Figure  1  is:  "Who  does  the 

adapting?"  A  partial  answer  to  this  question  involves  defining 


who  determines  the  situation,  attributes,  or  feedback.  Either 
the  user  must  specify  this  information  (perhaps  indirectly)  or 
the  aid  has  to  infer  it.  The  remainder  of  this  question  relates 
to  who  adapts  the  search  strategy.  This  may  be  accomplished 
either  by  the  user  specifying  adaptations  or  by  the  aid 
performing  the  adaptation.  Thus,  the  overall  answer  to  this 
second  question  can  range  from  solely  the  user,  to  a  combination 
of  user  and  aid,  to  solely  the  aid. 

The  third  question  in  Figure  1  is:  ’’How  is  the  adaptation 
done?"  Considering  the  "modifications”  involved,  the  answer  to 
this  question  depends  on  whether  decision  making  responsibility 
is  allocated  to  the  aid  or  partitioned  between  the  user  and  aid. 
For  "allocation”,  the  aid  would  infer  the  situation  or  attributes 
and  then,  adapt  the  search  keys  and/or  logic  and  finally,  suggest 
alternatives  to  the  user.  For  "partitioning",  the  user  would 
specify  the  situation,  attributes,  or  feedback  and  the  aid  would 
adapt  search  keys  and/or  logic  to  produce  suggested  alternatives. 

The  "models"  that  would  be  needed  to  perform  the  adaptation 
would  potentially  include  models  to  infer  situations  or 
attributes,  models  to  determine  appropriate  search  keys  and/or 
logic,  and  models  to  interpret  user  feedback  regarding  initially 
suggested  alternatives.  The  "measurements"  involved  would  be 
task  variables,  user  choices,  and  user  specifications.  Choices 
and  specifications  are  differentiated  here  to  contrast  users' 
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decisions  in  the  course  of  pursuing  a  task  and  users'  direct 
communications  with  an  aid. 


The  final  question  in  Figure  1  is:  "How  is  information 
transmitted?"  Communications  are  classified  as  direct,  indirect, 
and  inferred  along  one  dimension,  and  explicit  or  implicit  along 
the  other  dimension.  Direct  communications  include  measurements 
of  task  variables,  specifications  by  users,  and  information 
displayed  to  users.  Indirect  communications  include  users' 
decisions  (e.g.,  commands  employed  or  usage  of  suggested 
alternatives)  that  can  be  interpreted  by  the  aid  as  communication 
of  information.  Inferred  communications  would  include  inferences 
of  situations  or  attributes  based  on  direct  or  indirect 
communications . 


Typically,  direct  communications  are  explicit  while  indirect 
and  inferred  communications  are  implicit  although,  if  the 
situation  allows,  some  aspects  of  indirect  and  inferred 
communications  may  be  made  explicit  to  the  user.  This  can  be 
quite  important  if  the  user  must  detect  and  diagnose  errors  on 
the  part  of  the  aid. 


One  of  the  issues  raised  in  Figure  1  that  remains  to  be 
addressed  is  the  extent  to  which  adaptations  are  performed  online 
or  offline,  and  the  degree  to  which  the  aid  is  "pre-adapted"  via, 
for  example,  look-up  tables  of  situations/attributes  vs.  search 
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strategies.  The  extent  of  such  offline  adaptation  is  highly 
dependent  on  the  time  constraints  imposed  by  the  particular 
environment.  As  a  result,  while  the  decision  tasks  depicted  in 
Figure  2  represent  command  and  control  in  general,  the  feasible 
forms,  modes,  methods,  and  means  of  adaptation  depend  on  the  time 
constraints  operable  in  the  command  and  control  function  of 
interest.  Basically,  loose  time  constraints  allow  for 
substantial  online  user-aid  interaction  and  tight  time 
constraints  require  much  more  autonomy  on  the  part  of  the  aid. 

As  another  illustration  of  the  use  of  the  framework  proposed 
in  this  report,  consider  adaptive  aids  for  selection  among 
alternatives.  Relative  to  the  question  of  "What  is  adapted  to?", 
a  task-oriented  answer  would  include  the  set  of  alternatives  and 
selection  criteria  as  well  as  time-variations  of  alternatives  and 
criteria.  A  user-oriented  answer  to  this  question  would  include 
individual  differences  (e.g.,  level  of  training)  and  individual 
preferences  (e.g.,  attitude  toward  risk).  Considering  "Who  does 
the  adapting?",  either  the  user  must  specify  or  the  aid  must 
assess  variations  of  alternatives  and  criteria  as  well  as 
relevant  individual  differences  and  preferences.  Given  this 
information,  either  the  user  or  aid  could  perform  the  selection, 
although  aided  selection  is  certainly  the  norm  in  most  decision 
support  systems. 

Relative  to  "How  is  the  adaptation  done?",  modifications 


could  involve  allocation  or  partitioning.  For  allocation,  the 
aid  would  measure  or  infer  variations  and  differences,  and 
subsequently  adapt  the  structure  and/or  parameters  of  the 
selection  process.  For  partitioning,  the  user  would  specify 
variations  and/or  differences  and  the  aid  would  adapt 
accordingly.  The  models  of  interest  would  be  those  needed  for 
inferring  variations  and  differences  as  well  as  models  for 
multicriteria  allocation  problems.  Considering  "How  is  the 
information  transmitted?",  the  nature  of  the  answer  would  be 
similar  to  that  for  generation  tasks. 

As  a  final  illustration  of  the  use  of  the  proposed 
framework,  adaptive  aids  for  output  can  be  considered.  Relative 
to  "What  is  adapted  to?",  the  answer  is  the  plan  to  be 
implemented  and  time-variations  of  plan  parameters.  Considering 
"Who  does  the  adapting?",  either  the  user  specifies  the  plan  or 
the  aid  determines  it  via,  for  example,  a  data  base  of  procedures 
or  "scripts".  The  aid  would  then  monitor  plan  execution  and 
provide  some  form  of  feedback  relative  to  inconsistencies  and 
errors.  When  appropriate,  the  aid  would  assist  by  performing 
some  or  all  of  the  plan  implementation. 

The  questions  of  "How  is  the  adaptation  done?"  suggests 
modifications  in  terms  of  allocating  both  monitoring  and 
implementation  to  the  aid  or,  alternatively,  partitioning  the 
task  so  that  the  human  monitors  and  requests  assistance  whereupon 


the  aid  performs  the  portion  of  the  task  delegated.  The  models 
needed  here  would  be  those  associated  with  inferring  intentions 
(i.e.,  plans),  determining  expectations  and  comparing  them  with 
observations,  and  relating  workload,  urgency,  etc.  to  the 
possibility  of  achieving  task  objectives.  Considering  "How  is 
the  information  transmitted?",  direct  communications  would 
include  task  variables,  user  specifications,  and  displayed 
information.  Indirect  communications  would  include  user 
decisions  and  assessments  of  variables  such  as  workload. 
Inferred  communications  would  include  inferences  of  intentions 
(e.g.,  procedure  in  use)  and  requirements  for  assistance  (e.g., 
excessive  workload) . 

While  it  is  not  possible,  within  the  scope  of  this  report, 
to  elaborate  upon  all  of  the  aiding  issues  outlined  in  Figure  1 
as  they  relate  to  the  command  and  control  decision  making  tasks 
in  Figure  2,  the  above  discussion  should  provide  a  clear 
indication  of  how  this  mapping  is  done  and  the  way  in  which  this 
framework  identifies  important  issues  in  the  design  of  decision 
aids.  In  the  concluding  section  of  this  report,  recommendations 
for  research  that  have  emerged  from  considering  command  and 
control  within  this  framework  are  presented. 

Design  Principles  and  Guidelines 


As  noted  earlier,  a  number  of  authors  have  proposed 
principles  and/or  guidelines  for  the  design  of  decision  aids. 


Almost  all  of  this  material  relates  to  basic  human  factors  and 
human-computer  interaction  in  general.  Surprisingly  little 
focuses  on  decision  aiding  per  se.  Of  those  authors  who  do 
consider  the  unique  aspects  of  decision  making  that  should 
influence  design,  most  discussions  are  limited  to  recommendations 
of  tasks  that  should  be  aided.  Some  discussion  is  devoted  to 
when  aiding  is  needed  (e.g.,  during  peak  workload  periods  and  in 
multitask  situations).  There  is  also  a  fairly  standard  litany 
regarding  the  need  for  aids  to  be  adapted  to  specific  situations, 
individual  differences,  and  individual  preferences. 

Unfortunately,  very  few  documents  provide  principles  and/or 
guidelines  for  how  to  accomplish  the  design  of  aids.  The 
following  is  a  fairly  complete  list  of  principles  and  guidelines 
that  relate  specifically  to  decision  making,  particularly  in 
terms  of  the  tasks  shown  in  Figure  2. 

1 .  The  operating  principles  of  an  aid  should  be  sufficiently 
simple  for  users  to  understand  them. 

2.  The  structure  of  the  aiding  (e.g.,  data  base  and  decision 
process)  should  be  consistent  with  human  cognitive 
organization . 


3.  The  modes  of  failure  of  an  aid  should  be  detectable  and 
understandable  by  users. 


4*  Aids  should  be  easily  updated  and  modified  without 
compromising  system  integrity  or  users’  understanding  of 
their  operations. 

5.  Aids'  queries  to  users  and  information  displayed  should  be 


such  that  the  impacts  of  decision  making  biases  are 

minimized. 

6.  Recommendations  produced  by  aids  should  include  a  range  of 
alternatives  and  not  just  the  theoretical  optimum. 

7.  Aids  should  be  able  to  explain  the  basis  of  their 

recommendations. 

8.  Aids  should  be  such  that  users  still  feel  involved  in  the 
decision  making  process. 

Obviously,  this  list  is  very  general  and  it  is  not 
necessarily  straightforward  to  translate  these  principles  and 
guidelines  into  the  design  of  a  specific  aid.  While  it  might 
seem  that  this  translation  problem  mainly  reflects  an  inadequate 
knowledge  base,  an  equally  important  source  of  this  difficulty 
appears  to  be  a  fairly  poor  communication  process  between  basic 
research  in  human  decision  making  and  the  design  and 
implementation  of  decision  aids.  In  general,  it  seems  that  the 
state  of  the  art  in  understanding  human  decision  making  is  much 
more  advanced  than  reflected  by  available  design  principles  and 
guidelines . 
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RECOMMENDATIONS  AND  CONCLUSIONS 


The  framework  for  research  on  adaptive  decision  aids 
proposed  in  this  report  has  served  to  identify  a  wide  variety  of 
issues  that  should  be  addressed  in  the  design  and  development  of 
aids.  While  this  framework  may  cause  the  reader  to  feel  that 
design  and  development  is  rather  straightforward,  at  least 
conceptually,  this  is  really  not  the  case  since  many  of  the 
questions  raised  cannot,  as  yet,  be  answered  in  operationally 
useful  manners.  In  this  concluding  section,  four  areas  are 
discussed  where  research  is  needed  in  order  to  provide  the 
requisite  answers. 


User  Specification  of  Information 


Throughout  the  earlier  sections  of  this  report,  the 
possibility  of  having  users  explicitly  provide  information  to 
aids  was  noted.  It  was  suggested  that  users  could  specify 
situations,  attributes  of  alternatives,  individual  preferences, 
and  so  on.  The  obvious  difficulty  with  this  approach  is  the 
demands  that  it  places  on  the  user,  demands  that  potentially 
could  exceed  the  benefits  provided  by  an  aid.  Nevertheless, 
there  are  some  types  of  information  for  which  humans  are  the  best 
source,  particularly  because  of  human  knowledge  and  pattern 
recognition  abilities. 


Given  this  situation,  there  is  a  need  to  develop  more 


efficient  and  effective  methods  for  users  to  specify  information 
required  by  aids  in  order  to  adapt.  There  appear  to  be  three 
ways  in  which  to  deal  with  this  problem.  First,  some  information 
can  be  specified  in  advance,  perhaps  in  the  form  of  contingency 
tables.  Second,  methods  can  be  developed  for  identifying 
preference  structures,  for  example,  in  the  minimum  number  of 
queries  to  the  user.  Third,  various  coding  methods  can  be  used 
to  enable  the  user  to  communicate  in  the  minimum  number  of 
symbols. 

While  all  three  of  these  approaches  have  received  attention 
and  progress  is  being  made,  the  potential  for  user  specification 
of  all  information  required  by  aids  is  definitely  limited.  This 
is  particularly  true  when  time  constraints  are  tight.  Then,  it 
is  likely  that  aids  will  have  to  infer  information  in  order  to  be 
useful. 

Methods  of  Online  Inference 

Methods  are  needed  whereby  an  aid  can  employ  a  knowledge 
base,  as  well  as  time  histories  of  task  variables  and  user 
decisions,  to  infer  the  requisite  information.  The  inferences  of 
interest  encompass  the  situation,  attributes  of  alternatives, 
information  requirements,  intentions  or  plans,  individual 
differences,  individual  preferences,  and  variations  in  time  of 
each  of  these  types  of  information. 


While  this  may  appear  to  be  a  "tall  order,"  two  factors  make 
online  inference  more  tractable.  First,  and  foremost,  many 
command  and  control  domains  are  heavily  context-dominated. 
Therefore,  the  task  structure  is  fairly  well-defined  as  is  the 
information  that  is  relevant  to  different  phases  of  task. 
Further,  the  training  and,  to  at  least  a  slight  degree,  the 
preferences  of  users  are  often  doctrinally  defined. 

The  second  factor  of  interest  is  developments  in  advanced 
software  engineering  (i.e.,  artificial  intelligence)  where 
concepts  such  as  "frames,"  "scripts,"  and  "demons"  are  being  put 
to  practical  use.  Further,  as  noted  earlier,  the  "expert  system" 
idea  is  being  incorporated  in  various  decision  support  systems. 
Thus,  the  problem  context  seems  sufficiently  well-defined  and  the 
software  tools  appear  available  for  developing  methods  of  online 
inference.  Of  course,  this  is  not  meant  to  imply  that  such 
methods  are  totally  straightforward.  However,  there  is  a 
technology  base  to  support  the  needed  research  efforts. 

Methods  of  Online  Adaptation 

Given  that  an  aid  can  obtain  all  of  the  requisite 
information,  either  by  user  specification  or  by  inference, 
methods  of  adaptation  become  the  central  issue.  From  the 
analysis  outlined  in  earlier  sections  of  this  report,  it  appears 
to  the  authors  that  four  topics  deserve  particular  attention. 
The  first  involves  the  adaptation  of  computer-based  search 

strategies  to  situations,  attributes,  etc.  For  example,  a  method 
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is  needed  for  adapting  search  keys  and/or  logic  to 
time-variations  in  attributes  of  desirable  alternatives. 

A  second  topic  of  interest  involves  adapting  the  structure 
and  parameters  of  allocation  algorithms  so  that  aided  selection 
can  reflect  individual  preferences,  time-variations  of  resource 
constraints,  and  so  on.  This  topic  is  the  most  straightforward 
of  those  suggested  here;  it  is  included  because  of  the  ubiquity 
of  selection  aids. 

A  third  topic  for  research  is  methods  of  adapting  display 
transformations,  formats,  etc.  to  particular  situations  and 
individuals.  While  a  notable  amount  of  research  has  been 
directed  at  the  appropriateness  of  different  displays  to  various 
situations  and  types  of  individual,  the  results  of  this  work  have 
seldom  been  incorporated  in  online  adaptive  aids  and  subsequently 
experimentally  evaluated. 

The  fourth  and  final  topic  recommended  for  research  involves 
methods  of  adapting  the  level  of  aiding  (i.e.,  monitoring  vs. 
tranforming  vs.  partitioning  vs.  allocating)  to  time-variations 
of  tasks  as  a  function  of  individual  differences  and  preferences. 
Conceptually,  this  idea  is  reasonably  well-defined.  However,  due 
to  the  relative  novelty  of  this  idea,  the  research  base  needed  to 
support  the  design  of  adaptive  mechanisms  is  currently 
inadequate . 


A  Few  Crucial  Issues 

There  are  a  few  issues  that  should  heavily  influence  the  way 
in  which  the  research  topics  oulined  above  are  pursued.  Among 
these,  perhaps  the  most  important  issue  involves  the  degree  to 
which  there  should  be  explicit  communication  between  the  user  and 
aid.  At  one  extreme,  the  aid  could  inform  the  user  of  everything 
that  it  infers  and  the  resulting  adaptations.  At  the  other 
extreme,  the  aid  could  silently  toil  away  with  the  user  hardly 
aware  that  the  task  is  being  adaptively  manipulated  to  improve 
performance.  While  one  can  easily  envision  potential  advantages 
and  disadvantages  of  either  of  these  extremes,  there  is  little 
empirical  evidence  to  support  such  conjectures  directly. 
Research  is  needed  in  which  the  degree  of  explicit  communication 
is  systematically  manipulated,  perhaps  in  conjunction  with 
individual  differences  and  preferences. 

Another  crucial  issue  involves  development  and  testing  of 
online  methods  for  assessing  workload,  attention  allocation, 
planning  activity,  task  urgency,  and  other  important  information 
to  be  used  in  adapting  aids.  Measurement  techniques  in  these 
areas  would  reduce  the  need  for  inference  of  requisite 
information,  which  is  likely  to  improve  accuracy  and  simplify 
software.  While  a  variety  of  measurement  methods  in  these  areas 
has  been  reported  in  the  literature,  few  of  the  methods  have  been 
assessed  in  an  online  aiding  environment.  A  comprehensive  survey 


and  selected  testing  are  needed. 

Finally,  the  resolution  of  all  of  the  issues  discussed  in 
this  report  is  very  dependent  on  the  time  constraints  existing  in 
the  domain  of  interest.  Since  such  constraints  differ  greatly 
for  the  various  command  and  control  domains  noted  in  this  report, 
it  is  very  important  that  this  variable  be  considered  when 
interpreting  the  results  of  any  research  in  terms  of  their 
applicability  across  domains.  Such  interpretations  would  be 
facilitated  by  a  research  effort  which  systematically  manipulated 
time  constraints  to  assess  their  effects  on  the  various  factors 
associated  with  the  design  of  adaptive  decision  aids. 
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